Traditional congestion control algorithms were designed with a hardwired heuristic mapping between packet-level events and predefined control actions in response to these events, and may fail to satisfy all the desirable performance goals as a result. In this paper, we seek to reconsider these fundamental goals in congestion control, and propose Eagle, a new congestion control algorithm to refine existing heuristics. Eagle takes advantage of expert knowledge from an existing algorithm, and uses deep reinforcement learning (DRL) to train a generalized model with the hope of learning from an expert. Learning by trial-and-error may not be as efficient as imitating a teacher; by the same token, DRL alone is not enough to guarantee good performance. In Eagle, we seek help from an expert congestion control algorithm, BBR, to help us train a long-short term memory (LSTM) neural network in the DRL agent, with the hope of making decisions that can be as good as or even better than the expert. With an extensive array of experiments, we discovered that Eagle is able to match and even outperform the performance of its teacher, and outperformed a large number of recent congestion control algorithms by a considerable margin.

Network alignment (NA) -- i.e., linking entities from different networks (also known as identity linkage) -- is a fundamental problem in many application domains. Recent advances in deep graph learning have inspired various auspicious approaches for tackling the NA problem. However, most of the existing works suffer from efficiency and generalization, due to complexities and redundant computations. We approach the NA from a different perspective, tackling it via meta-learning in a semi-supervised manner, and propose an effective and efficient approach called Meta-NA -- a novel, conceptually simple, flexible, and general framework. Specifically, we reformulate NA as a one-shot classification problem and address it with a graph meta-learning framework. Meta-NA exploits the meta-metric learning from known anchor nodes to obtain latent priors for linking unknown anchor nodes. It contains multiple sub-networks corresponding to multiple graphs, learning a unified metric space, where one can easily link entities across different graphs. In addition to the performance lift, Meta-NA greatly improves the anchor linking generalization, significantly reduces the computational overheads, and is easily extendable to multi-network alignment scenarios. Extensive experiments conducted on three real-world datasets demonstrate the superiority of Meta-NA over several state-of-the-art baselines in terms of both alignment accuracy and learning efficiency.

Understanding the nature of user-device interactions (e.g., who is using the device and what he/she is doing with it) is critical to many applications including time management, user profiles, and privacy protection. However, in scenarios where mobile devices are shared among family members or multiple employees in a company, conventional account-based statistics are not meaningful. This poses an even bigger problem when dealing with sensitive data. Moreover, fingerprint readers and front-facing cameras were not designed to continuously identify users. In this study, we developed MagPrint, a novel approach to fingerprint users based on unique patterns in the electromagnetic (EM) signals associated with the specific use patterns of users. Initial experiments showed that time-varying EM patterns are unique to individual users. They are also temporally and spatially consistent, which makes them suitable for fingerprinting. MagPrint has a number of advantages over existing schemes: i) Non-intrusive fingerprinting, ii) implementation using a small and easy-to-deploy device, and iii) high accuracy thanks to the proposed classification algorithm. In experiments involving 30 users, MagPrint achieves 94.3% accuracy in classifying users from these traces, which represents an 10.9% improvement over the state-of-the-art classification method.

Recent years have seen a rapidly increasing traffic demand of HTTP-based high-quality live video streaming. The surging traffic demand and the realtime property of live videos make it challenging for the content delivery networks (CDNs) to guarantee the Quality-of-Experiences (QoE) of viewers. Initial video segment (IVS) of live streaming plays an important role for the QoE of viewers, particularly when they require fast join and smooth view experience. State-of-the-art research on this regard estimates network throughput for each viewer and thus may incur a large overhead that offsets the benefit. To tackle the problem, we propose Rldish, a scheme deployed at the edge CDN server, to dynamically select a suitable IVS for new live viewers based on Reinforcement Learning (RL). Rldish is transparent to both the client and streaming server. It collects the real-time QoE observations from the edge without any client-side assistance, then uses these QoE observations as realtime rewards in RL. We deploy Rldish as a virtualized network function in a real HTTP cache server, and evaluate its performance using streaming servers distributed over the world. Our experiments show that Rldish improves the state-of-the-art IVS selection scheme w.r.t. the average QoE of live viewers by up to 22%.

Deep learning shows great promise in providing more intelligence to the mobile web, but insufficient infrastructure, heavy models, and intensive computation limit the use of deep learning in mobile web applications. In this paper, we present DeepAdapter, a collaborative framework that ties the mobile web with an edge server and a remote cloud server to allow executing deep learning on the mobile web with lower processing latency, lower mobile energy, and higher system throughput. DeepAdapter provides a context-aware pruning algorithm that incorporates latency, the network condition and the computing capacity of mobile device to fit the resource constraints of the mobile web better. It also provides a model cache update mechanism improving the model request hit rate for mobile web users. At runtime, it matches an appropriate model with the mobile web user and provides a collaborative mechanism to ensure accuracy. Our results show that DeepAdapter decreases average latency by 1.33x, reduces average mobile energy consumption by 1.4x, and improves system throughput by 2.1x without a loss in accuracy. Its context-aware pruning algorithm also improves inference accuracy by up to 0.3% with a smaller and faster model.

Deep reinforcement learning (DRL) may be leveraged to empower wireless devices to "sense" current spectrum and network conditions and "react" in real time by either exploiting known optimal actions or exploring new actions. Yet, understanding whether real-time DRL can be at all applied in the resource-challenged embedded IoT domain still remains mostly uncharted territory. This paper bridges the existing gap between the extensive theoretical research on wireless DRL and its system-level applications by presenting Deep Wireless Embedded Reinforcement Learning (DeepWiERL), a general-purpose, hybrid software/hardware DRL framework specifically tailored for embedded IoT wireless devices. DeepWiERL provides abstractions, circuits, software structures and drivers to support the training and real-time execution of state-of-the-art DRL algorithms on the device's hardware. Moreover, DeepWiERL includes a novel supervised DRL model selection and bootstrap (S-DMSB) technique that leverages transfer learning and high-level synthesis (HLS) circuit design to orchestrate a neural network that satisfies hardware and application throughput constraints and improves the DRL algorithm convergence. Experimental evaluation shows that DeepWiERL supports 16x data rate and consumes 14x less energy than a software-based implementation, and that (iii) S-DMSB may improve the DRL convergence time by 6x and increase the reward by 45% if prior channel knowledge is available.

Deep neural networks (DNN) are the de-facto solution behind many intelligent applications of today, ranging from machine translation to autonomous driving. DNNs are accurate but resource-intensive, especially for embedded devices such as smartphones and smart objects in the Internet of Things. To overcome the related resource constraints, DNN inference is generally offloaded to the edge or to the cloud. This is accomplished by partitioning the DNN and distributing computations at the two different ends. However, existing solutions simply split the DNN into two parts, one running locally or at the edge, and the other one in the cloud. In contrast, this article proposes a solution to divide a DNN in multiple partitions that can be processed locally by end devices or offloaded to one or multiple powerful nodes, such as in fog networks. The proposed solution includes both an adaptive DNN partitioning scheme and a distributed algorithm to offload computations based on a matching game approach. Results obtained by using a self-driving car dataset and several DNN benchmarks show that the proposed solution significantly reduces the total latency for DNN inference compared to other distributed approaches and is 2.6 to 4.2 times faster than the state of the art.

Large-scale spatial data such as air quality, thermal conditions and location signatures play a vital role in a variety of applications. Collecting such data manually can be tedious and labour intensive. With the advancement of robotic technologies, it is feasible to automate such tasks using mobile robots with sensing and navigation capabilities. However, due to limited battery lifetime and scarcity of charging stations, it is important to plan paths for the robots that maximize the utility of data collection, also known as the informative path planning (IPP) problem. In this paper, we propose a novel IPP algorithm using reinforcement learning (RL). A constrained exploration and exploitation strategy is designed to address the unique challenges of IPP, and is shown to have fast convergence and better optimality than a classical reinforcement learning approach. Extensive experiments using real-world measurement data demonstrate that the proposed algorithm outperforms state-of-the-art algorithms in most test cases. Interestingly, unlike existing solutions that have to be re-executed when any input parameter changes, our RL-based solution allows a degree of transferability across different problem instances.

Mobile CrowdSensing (MCS) is a promising paradigm that recruits users to cooperatively perform various sensing tasks. In most realistic scenarios, users dynamically participate in MCS, and hence, we should recruit them in an online manner. In general, we prefer to recruit a user who can make the maximum contribution at the least cost, especially when the recruitment budget is limited. The existing strategies usually formulate the user recruitment as the budgeted optimal stopping problem, while we argue that not only the budget but also the time constraints can greatly influence the recruitment performance. In this paper, we propose a dynamic user recruitment strategy with truthful pricing to address the online user recruitment problem under the budget and time constraints. To deal with the two constraints, we first estimate the number of users to be recruited and then recruit them in segments. Furthermore, to correct estimation errors and utilize newly obtained information, we dynamically re-adjust the recruiting strategy. Finally, an online pricing mechanism is lightly built into the proposed user recruitment strategy. Extensive experiments on three real-world data sets validate the proposed online user recruitment strategy, which can effectively improve the number of completed tasks under the budget and time constraints.

With the popularity of unmanned aerial vehicles (UAV) and driverless cars, vehicular crowdsensing (VCS) becomes increasingly widely-used by taking advantage of their high-precision sensors and durability in harsh environments. Since abrupt sensing tasks usually cannot be prepared beforehand, we need a generic control logic fit-for-use all tasks which are similar in nature, but different in their own settings like Point-of-Interest (PoI) distributions. The objectives include to simultaneously maximize the data collection amount, geographic fairness, and minimize the energy consumption of all vehicles for all tasks, which usually cannot be explicitly expressed in a closed-form equation, thus not tractable as an optimization problem. In this paper, we propose a deep reinforcement learning (DRL)-based centralized control, distributed execution framework for multi-task-oriented VCS, called "DRL-MTVCS". It includes an asynchronous architecture with spatiotemporal state information modeling, multi-task-oriented value estimates by adaptive normalization, and auxiliary vehicle action exploration by pixel control. We compare with three baselines, and results show that DRL-MTVCS outperforms all others in terms of energy efficiency when varying different numbers of tasks, vehicles, charging stations and sensing ranges.

Truth discovery is an effective tool to unearth truthful answers in crowdsourced question answering systems. Incentive mechanisms are necessary in such systems to stimulate worker participation. However, most of existing incentive mechanisms only consider compensating workers' resource cost, while the cost incurred by potential privacy leakage has been rarely incorporated. More importantly, to the best of our knowledge, how to provide personalized payments for workers with different privacy demands remains uninvestigated thus far. In this paper, we propose a contract-based personalized privacy-preserving incentive mechanism for truth discovery in crowdsourced question answering systems, named PINTION, which provides personalized payments for workers with different privacy demands as a compensation for privacy cost, while ensuring accurate truth discovery. The basic idea is that each worker chooses to sign a contract with the platform, which specifies a privacy-preserving level (PPL) and a payment, and then submits perturbed answers with that PPL in return for that payment. Specifically, we respectively design a set of optimal contracts under both complete and incomplete information models, which could maximize the truth discovery accuracy, while satisfying the budget feasibility, individual rationality and incentive compatibility properties. Experiments on both synthetic and real-world datasets validate the feasibility and effectiveness of PINTION.

Recently, data-driven prediction strategies have shown the potential of shepherding the optimization strategies for end viewer's Quality-of-Experience in practical streaming applications. While current prediction-based designs have largely focused on optimizing the last-mile, i.e., viewer-side, which still have several limits as they: (1) need the real-time feedback from viewers to improve the prediction accuracy; (2) need quick responses to guarantee the effectiveness of optimization strategies in the future. Thanks to the emerged crowdsourced livecast services, e.g., Twitch.tv, we for the first time exploit the opportunity to realize the long-term prediction and optimization with the assistance derived from the first-mile, i.e., source broadcasters.

In this paper, we propose a novel framework \textit{CastFlag}, which analyzes the broadcasters' operations and interactions, predicts the key events, and optimizes the ingesting, transcoding, and distributing stages in corresponding live streams, even before the encoding stage. Taking the most popular eSports gamecast as an example, we illustrate the effectiveness of this framework in the game highlight (i.e., key event) prediction and transcoding workload allocation. The trace-driven evaluation shows the superiority of CastFlag as it: (1) improves prediction accuracy over other learning-based approaches by up to 30%; (2) achieves average 10% decrease of the transcoding latency at less cost.

In this paper we study a team orienteering problem, which is to find service paths for multiple vehicles in a network such that the profit sum of serving the nodes in the paths is maximized, subject to the cost budget of each vehicle. This problem has many potential applications in IoT and smart cities, such as dispatching energy-constrained mobile chargers to charge as many energy-critical sensors as possible to prolong the network lifetime. In this paper, we first formulate the team orienteering problem, where different vehicles are different types, each node can be served by multiple vehicles, and the profit of serving the node is a submodular function of the number of vehicles serving it. We then propose a novel 0.32-approximation algorithm for the problem. In addition, for a special team orienteering problem with the same type of vehicles and the profits of serving a node once and multiple times being the same, we devise an improved approximation algorithm. Finally, we evaluate the proposed algorithms with simulation experiments, and the results of which are very promising. Precisely, the profit sums delivered by the proposed algorithms are approximately 12.5% to 17.5% higher than those by existing algorithms.

Electric autonomous vehicles provide a promising solution to the traffic congestion and air pollution problems in future smart cities. Considering intensive energy consumption, charging becomes of paramount importance to sustain the operation of these systems. Motivated by the innovations in renewable energy harvesting, we leverage solar energy to power autonomous vehicles via charging stations and solar-harvesting rooftops, and design a framework that optimizes the operation of these systems from end to end. With a fixed budget, our framework first optimizes the locations of charging stations based on historical spatial-temporal solar energy distribution and usage patterns, achieving (2+\epsilon) factor to the optimal. Then a stochastic algorithm is proposed to update the locations online to adapt to any shift in the distribution. Based on the deployment, a strategy is developed to assign energy requests in order to minimize their traveling distance to stations while not depleting their energy storage. Equipped with extra harvesting capability, we also optimize route planning to achieve a reasonable balance between energy consumed and harvested en-route. Our extensive simulations demonstrate the algorithm can approach the optimal solution within 10-15% approximation error, and improve the operating range of vehicles by up to 2-3 times compared to other competitive strategies.

Large scale deployment of connected vehicles with cooperative sensing technologies increases the demand on the vehicular communication spectrum band in 5.9 GHz allocated for exchange of safety messages. To support high data rates needed by such applications, the millimeter-wave (mmWave) automotive radar spectrum at 76-81 GHz spectrum can be utilized for communication. For this purpose, joint automotive radar-communication (JARC) system designs are proposed in the literature to perform both functions using the same waveform. However, employing large bandwidth at mmWave spectrum deteriorates the performance of both radar and communication functions due to frequency-selectivity. In this paper, we address the optimal joint waveform design problem for wideband JARC systems that use Orthogonal Frequency-Division Multiplexing (OFDM) signal. We show that the problem is a non-convex Quadratically Constrained Quadratic Fractional Programming (QCQFP) problem, which is known to be NP-hard. Existing approaches to solve QCQFP include Semidefinite Relaxation (SDR) and randomization approaches, which have high time complexity. Instead, we propose an approximation method to solve QCQFP more efficiently by leveraging structured matrices in the quadratic fractional objective function. Finally, we evaluate the efficacy of the proposed approach through numerical results.

Wireless communications for status update are becoming increasingly important, especially for machine-type control applications. Existing work has been mainly focused on Age of Information (AoI) optimizations. In this paper, a status-aware predictive wireless interface design, networking and implementation are presented which aim to minimize the status recovery error of a wireless networked system by leveraging online status model predictions. Two critical issues of predictive status update are addressed: practicality and usefulness. Link-level experiments on a Software-Defined-Radio (SDR) testbed are conducted and test results show that the proposed design can significantly reduce the number of wireless transmissions while maintaining a low status recovery error. A Status-aware Multi-Agent Reinforcement learning neTworking solution (SMART) is proposed to dynamically and autonomously control the transmit decisions of devices in an ad hoc network based on their individual statuses. System-level simulations of a multi dense platooning scenario are carried out on a road traffic simulator. Results show that the proposed schemes can greatly improve the platooning control performance in terms of the minimum safe distance between successive vehicles, in comparison with the AoI-optimized status-unaware and communication latency-optimized schemes---this demonstrates the usefulness of our proposed status update schemes in a real-world application.